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1. Introduction 

Since the introduction by Slutsky [13], moving average models have played a signifi- 
cant role in time series analysis, especially in finance and economics. The models have 
been extended to include measurable (nonlinear) functions of independent and identi- 
cally distributed random variables, representing unobservable and purely random im- 
pulses, for example, Robinson [12]. The characterizing feature of these models is the 
cut-off of the auto-covariance functions when they exist, implying that they are models 
of short memory. Another interesting feature of these models is the homogeneity of the 
random impulses, free from any feedback in the generating mechanism. Now, Slutsky 
developed these models in an economic context; the random impulses may correspond to 
unobservable political factors. In such a context, as well as in other contexts for which 
these models are relevant (e.g., business studies), it can be argued that feedback is often 
present: political decisions are often predicated on economic conditions. One simple way 
to incorporate feedback in these models is through the notion of thresholds, that is, on-off 
feedback controllers. 

Since Tong [14] initiated the threshold notion in time series modelling, the notion has 
been extensively used in the literature, especially for the threshold autorcgrcssivc (TAR) 
or TAR-type models. For these models, some basic and probabilistic properties were 
given in Chan et al. [3] and Chan and Tong [4]. More related results can be found in 
An and Huang [1], Brockwcll et al. [2], Chen and Tsay [5], Clinc and Pu [6, 7], Ling [8], 
Ling et al. [9], Liu and Susko [10] and Lu [11], among others. A fairly comprehensive 
review of threshold models is available in Tong [15] and a selective survey of the history 
of threshold models is given by Tong [16]. 
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However, most work to-date on the threshold model has primarily concentrated on 
the TAR or the TAR-type model. The threshold moving average (TMA) model, that 
is a moving average model with a simple on-off feedback control mechanism, has not 
attracted as much attention. As far as we know, only a few results are available for the 
TMA model. Brockwcll et al. [2] investigated a threshold autoregressive and moving- 
average (TARMA) model and obtained a strictly stationary and ergodic solution to the 
model when the MA part does not contain any threshold component. Unfortunately, 
their TARMA model does not cover the TMA model as a special case. Using the Markov 
chain theory, Liu and Susko [10] provided the existence of the strictly stationary solution 
to the TMA model without any restriction on the coefficients. However, they neither 
gave an explicit form of the solution nor proved the ergodicity. A similar result can be 
found in Ling [8]. Ling et al. [9] gave a sufficient condition for the ergodicity of the 
solution for a first order TMA model under some restrictive conditions. These results 
have been extended to the first-order TMA model with more than two regimes. However, 
the uniqueness and the ergodicity of the solution are still open problems for higher-order 
TMA models. 

In this paper, we use a different approach to study the TMA model without resort- 
ing to the Markov chain theory. Note that the TMA model involves a feedback control 
mechanism. An intuitive and simple idea is to seek a closed form of the solution in terms 
of the above mechanism, which is expressible as an indicator function. We can show that 
for the TMA model there always exists a unique strictly stationary and ergodic solution 
without any restriction on the coefficients of the TMA model. More importantly, for the 
first time in the literature, an explicit/closed form of the solution is derived. In addi- 
tion, for the correlation structure, we show that the ACF (when it exists) of the TMA 
model typically does not cut off. In fact, it has a much richer structure. For example, 
it can exhibit almost long memory, although it generally decays at an exponential rate. 
Furthermore, the difference between the joint two-dimensional distribution and the cor- 
responding product of its marginal distributions also decays to zero at an exponential 
rate as the lag tends to infinity. 

The rest of the paper is organized as follows. Section 2 discusses the strict stationarity 
and ergodicity of the TMA model. Section 3 studies the asymptotic behaviour of the 
ACF of the TMA model and other correlation structure. We conclude in Section 4. All 
proofs of the theorems are relegated to the Appendix. 



2. Stationarity and ergodicity of TMA(cjf) models 



We first consider a TMA(<j) model which satisfies the following equation: 




if V: 



if 2A 



'n 



(2.1) 



Moving- average models 



3 



where {e„} is a sequence of i.i.d. random variables. Here, q and d are positive integers, 
r € R, the real line, is the threshold parameter, and fii,^,^ an d ipi, * — 1, - - - , 9, are 
real coefficients. 

For the sake of simplicity, we adopt the following notation: 

U n = l(a„<r) and W„ = l(b n < r) - l(o„ < r), 

where l(-) is an indicator function, 

9 9 



A*2 



and &n 



(2.2) 



i=l i=l 

The following theorem gives the strict stationarity and ergodicity of model (2.1). 



Theorem 2.1. Suppose that {e n } is a sequence of i.i.d. random variables with P(a„ < r, 
b n < f) + W{a-n > T,b n > r) 7^ 0. Then y n has a unique strictly stationary and ergodic 
solution expressed by 



where 



i.=i 



oo 
3=1 



(/il - jtl 2 ) + £(0i - lpi)Zr, 



a.s. 



<s=l / 



in L 1 and a.s. 



If ei has a strictly and continuously positive density on K (e.g., normal, Student's t v or 
double exponential distribution), then P(a„ <r,b n < r) +P(a„ > r,b n > r) ^ 0. The basic 
idea for Theorem 2.1 is a direct and concrete expression in terms of l(y n -d < r), without 
resorting to the Markov chain theory. Theorem 2.1 shows that the TMA(g) model is 
always stationary and ergodic as is the MA(g) model. 



3. The ACF of TMA(g) models 

The ACF plays a crucial role in studying the correlation structure of weakly stationary 
time series. It is well known that for a causal ARMA(p, q) model, its ACF pk goes to 
zero at an exponential rate as k diverges to infinity. The exact formula for ACF can be 
obtained although its closed form is not compact. However, for a general nonlinear time 
series model, it is rather difficult to obtain an exact formula for the ACF and to study 
the asymptotic behaviour. Additionally, the notion of memory, short or long, is closely 
associated with the ACF. One significant fact is that a causal ARMA(p, q) model is short- 
memory. For a general nonlinear time series model, due to its complicated structure, there 
is no universally accepted criterion for determining whether or not it is short-memory. 
As for some specific time series model, an ad hoc approach is usually adopted. 

One important characteristic of the MA(g) model is that its ACF cuts off after lag q. 
Interestingly, this property is not generally inherited by the TMA model; this is not sur- 
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prising theoretically because the TMA model involves some nonlinear feedback. Another 
interesting fact is that although a TMA model is generally short-memory, in some cases 
it can exhibit some almost long-memory phenomena; see Example 3.3. The following 
theorem characterizes the ACF of model (2.1). 

Theorem 3.1. Suppose that the condition in Theorem 2.1 is satisfied and E|ei| 2 < oo. 
Then there exists a constant p £ (0, 1) such that p k = 0(p k ). 

Theorem 3.1 indicates that the TMA model (2.1) is short-memory. The next theo- 
rem describes the relationship between the two-dimensional joint distribution and the 
corresponding marginal distributions. 

Theorem 3.2. Suppose that {e n } is i.i.d. random variables having a continuously, 
boundedly and strictly positive density. Then, for any ti,wel and k > 1, there exists 
a constant p G (0, 1) such that 



Actually, Theorem 3.2 still holds for Cov(l(wi < yo < U2),l(vi < y k < v 2 )) where 
—00 < ui < U2 < 00 and —00 <v\ < v 2 < 00. Next, we consider some special TMA models 
and study their ACFs as well as some other properties. 

Example 3.1. Suppose that y n is defined as 



where {e n } satisfies the condition in Theorem 2.1 with mean and finite variance a 2 . 

This example can also be regarded as a special case of the TAR model, which was 
studied in Tong [15], Question 29, page 212. By calculation, we have the ACF of {y n } 



where A fc = E[e„_ fc l(y„_i < r)], j8 = [G(r - pi) - G(r - p 2 )] e (-1, 1) and S = G(r -p 2 )/ 
[1 — G(r — pi) + G(r — p2)]- Here, G(x) is the distribution function of e±. 



Clearly, the ACF does not possess the cut-off property except for pi = p 2 - Generally, 
Pk decays exponentially since = 0(p k ) for some p € (0, 1) by the proof of Theorem 3.2. 
In the nonlinear time series literature, the search for a nonlinear AR model with long 
memory has been largely in vain. Against this background, it is interesting to note that 
as pi — > 00 and p 2 —00, pk can exhibit almost long memory in that pk can be made to 
decay arbitrarily slowly. Note that Example 3.1 can be driven by a white noise process 
with a thin tailed distribution. The skewness and the kurtosis of y n are also available 
explicitly and interesting. Specifically, 



|P(l/o < u, y k < v) - P(y < u)F(y k <v)\ = 0(p k ). 




if 2/n-i < r, 
if y„_i > r, 



{pi - p 2 )X k + (px - P2) 2 S (1 - S )/3 
o- 2 + {p 1 -p 2 ) 2 6 (l-S ) 



Pk = 



for k > 1 



skewness = 



Eel + (p 1 -p 2 f(S -3S 2 + 26 3 ) 
[a 2 + (/i 1 -/i 2 ) 2 <S (l- ( 5o)] 3 / 2 
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Figure 1. The skewness (left) and the kurtosis (right) of y n as functions of r when ei is standard 
normal. 



and 



kurtosis : 



Sej + Go-Hfn - fi 2 ) 2 6o(l - S Q ) + (jh - (i 2 )\5y - 4J 2 + 6£g - 3<%) 



respectively. The impact of the threshold parameter r is related to the bi-modality of the 
marginal density, which can be established by simple calculation. When e\ is standard 
normal and ((11,(12) = (4,-1), Figure 1 shows the skewness and the kurtosis of y n as 
functions of r. 

Example 3.2. Suppose that {y n } follows a TMA(l) model without drift: 

e n + (f>e n -i, iiy n -2<r, 
e n + i/je n -i, \ly n -2>r, 

where {e„} satisfies the condition in Theorem 2.1, having zero mean and finite variance. 

After simple calculation, we have the ACF 

%I)+{4>-%I))q 



0, 



if k = l, 
if k > 1, 



where g = P(e 2 + ipei < r)/[P(e 2 + $e\ > r) + P(e 2 + V^i < r)} e [0, 1). 
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Figure 2. The sample ACFs of model (3.1) 
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This example shows that for some special TMA(g) model, the ACF may be cut off 
after lag q. In particular, if <j> = ip, then the ACF coincides with that of the classical 
linear MA(1) model. Unfortunately, for general TMA models with d < q, there are no 
explicit expressions available for the ACFs due to the extremely complicated dependence 
of yt-d on {e t -j,d <j<q}- However, we can obtain the sample ACFs of TMA models 
by simulation. 

Example 3.3. Suppose that {y n } follows the TMA(l) model: 

f 5 + e„ + 0.2e„_i, if y n -i< 0.5, , . 

2/ ' 1 ~l-3 + e„ + 0.8e n _ 1 , if y n _ x > 0.5, 

where {e n } is i.i.d. standard normal. 

This model produces a time series that mimics a unit root and long memory In Fig- 
ure 2, the sample ACF of model (3.1) decays slowly, although model (3.1) is stationary. 



4. Concluding remarks 

Conventional moving average models, whether linear or nonlinear, assume absence of any 
feedback control mechanism. This paper shows that the introduction of simple feedback 
can enrich the structure of moving average models. For example, their ACF need not cut 
off but can now exhibit (near) long memory. Their distributions can be leptokurtic even 
when driven by Gaussian white noise. In nonlinear time series modeling, moving average 
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models have been overshadowed by autoregressive models. Our study suggests that, by 
introducing a simple feedback mechanism, the notion of moving average possesses some 
unexpected properties beyond the shadow. 



Appendix A: Proofs of theorems 
A.l. Proof of Theorem 2.1 

From model (2.1), l(y n < r) = U n + W n l(y n -d < r). Iterating k > 1 steps, we have 



fc-i 

i(y n <r) = J2 

3=0 



'3-1 \ 

| W n - S d U n - jd 



>k-l 



\ W n - ld l{y n - k d < r) 



with the convention Y[q 1 — 1- Let 



fe-i 



(3-1 \ 

\ W n - Sd Un-jd 



j=o L \s=o / 

For given d and q, there exists a unique nonnegative integer m such that md < max((i, q + 
1) < (m + Let <5 = E|lUi|. Under the condition in Theorem 2.1, it is not difficult to 
prove that < 6 < 1. Observing that both {U n } and {W„} are g-dependent sequences, 
we can extract an independent subsequence {W n -j( m +i)d-> 3 — 0,1,..., [^-pj]} from the 
sequence {W n -id, i = 0, 1,2, . . ., k — 1}, where [a] denotes the integral part of a. Since 
\U n \ < 1 and \ W n \ < 1, it yields that 





/k-1 




E 




U n -kd 




\i=0 





implying 



<i=0 / 



< (E|Wi|) [(fc - 1)/(m+1)1 , 



< = (m + 1) ^ <5 fe < oo. 



Using the above inequalities, we can prove that E|a„ ;S — Q! n f| — ^ as s,t — > oo for each 
fixed n. By the Cauchy criterion, a rh k converges in L 1 as k — > oo. Write the limit as 



(3-1 > 
] W n - sd 



Un- 



3=0 L \s=0 / 

Applying the inequalities above again, it is easy to get 



£EK, fc - a n \ <Y,Y, sKj - 1)/(m+1)] < oo. 
fe=i k=ij=k 
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yielding that 



lim a n ,k = OL n , 

k— J-oo 



in L 1 and a.s. 



Furthermore, recall that U n = l(a n < r) and W n = l(b n < r) — l(a n < r), where a n 
and b n are defined in (2.2), we have the iterative sequence: a n .\ = C/„ and 

u n ,k = U n + W n a n -d,k-i = (1 - an-d,fc-i)l(a« < r) + a„_ d!fc _il(& n < r) 

for each n and k > 1. Note that a n ,fc and a n -d,k have the same distribution for fixed fc 
since the error {e^} is i.i.d. By induction over k, we have that a n ,k only takes two values 
and 1 a.s. since a nt i only takes and 1. Thus, a n at most takes two values and 1 a.s., 
namely, a„ = l(a n = 1) a.s. Define a new sequence {S n } 



»=i 



(Mi - M2) +y"X<fc - -0j)e n - 4 



i=i 



By simple calculation, we have 

< r) = l(a„ < r)l(a„_ d = 0) + l(b n < r)l(a n _ d = 1) 
= C/ n + W„l(a n _ d = l) 
= J7 n + W n a n -d = ot n , a.s. 



Hence, 



S n =112 + e„ + J~] ipje r 



i=l 



(Ml - M2) +y^(0i - V'<) e ri 



l(S n -d<r), a.s. 



Thus, {S'n} is the solution of model (2.1) which is strictly stationary and ergodic. 
To uniqueness, suppose that S n is a solution to model (2.1), then 

l(S n <r)=U n + W n l(S n - d <r). 

Iterating the above equation, one can get for k > 1 

/fe-i \ 



1(S„ <r)= a n . k + l[ W n - ld l(S n -kd < 



We can show that the second term of the previous equation converges to zero a.s. Thus, 
we have l(S n <r)= a n a.s. Therefore, 



S n = P-2 + e n + 2J i'i 



a„_d, a.s. 



that is, = S„ a.s. The proof is complete. 
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The notations a n and b n are denned by (2.2), m and 8 are the same as those in the proof of 
Theorem 2.1. From Theorem 2.1, we have y n = a n + (b n — a n )a n -d- For n > (m + 2)d + q, 
we decompose a n -d into two parts 



[(„-g)/d]-l 

E 

/i+/ 2 . 



\ W n ^ sd J Un-jd 



E 

j=[(n-g)/d] 



V.s=l / 



Clearly, ij <= J 7 ™^ and J 2 <E J 7 ™^, where 7£ = a(e m , . . . , e„). By calculation, wc have 
for n > (m + 2)d + q 



I Cov(y ,ZM)| < X! 

j=[(n-g)/d] 



Em |W n _ srf | 
. \s=l 

F [0-l)/(™+l)] r 



1/2 



< E ^ 

j=[(n-«)/«d 

H(m + 1) ^[(n-q-d)/(d(m+l))]-l 



[E(b n ~a n ) 2 (y -Ey Q ) 2 } 1/2 



(b n -a n ) 2 ]^ 2 [E(y Q -Ey Q ) 2 ]^ 2 



< 



i-Vs 



by Holder's inequality, the boundcdncss of W n and f/ n , and the independence of {6 n , a n } 
and yo j where 



|/ii-H + (Ee 2 ) 1/2 X|&-^ 



M + M + (Ee 2 ) 1 / 2 £(|^| + |V>i|) 



Thus, the conclusion holds. 



A.3. Proof of Theorem 3.2 

Let x n = a n + (b n - a n )h. Then y n -x n = (b n -a n )h- Clearly, x n € F% and E\y n -x n \ = 
0(p n ) for large enough n, where p G (0, 1). So, using the independence of x n and yo, for 
large enough n, we have 

|P(yo < u, y n < v) - P(y < u)f>(y n < v) \ 
= |E{[l(y„ < v) - \{x n < v)][l(yo < u) - El(y < u)}}\ 
<E\l(y n <v)-l(x n <v)\. 

On noting the independence between e„ and e„_i, where e„_i = ^2 + SiLiV'i^-i + 
(6„ — a„)a„_d, the density of y n is /j,(a;) = L /i(a; — y) dG g (y), where /i(a;) is the density 
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function of ej and G g (y) is the distribution function of e„_i. On using the property of 
convolution, f y (x) is continuous and bounded. Write ||/y||oo = m ax{|/. y (a;)|: i£l}. On 
the one hand, using the following inequality 

\l(x<t) - l(y<t)\l(\x-y\ <e)<l{t-e<x<t + e), 

we can get 

E[|l(y„ < v) - l(x n < v)\l(\y n -x n \< e)] < P(« - e < y n < v + e) < 2||/ ! ,|| 00 e. 
On the other hand, using Markov's inequality, we have 

E{\l(y n <v)- l(x n < v)\l(\y n -x„\> e)} < El(\y n - x n \ > e)} < E~ l E\y n -x n \. 
Choosing e = O( j o"/ 2 ), we can obtain 

|P(yo < u,y n < v)-¥(y < u)F(y n <v)\ = 0{ P n ' 2 ). 
Hence, the result holds. 
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